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Abstract 

As  part  of  the  Navy’s  Decision  Centered  Design  (DCD)  program1,  preliminary  Cognitive  Task 
Analyses  (CTA)  were  performed  on  Joint  Operations  Center  (JOC)  personnel  serving  aboard 
command  and  control  ships  under  the  command  of  a  Joint  Task  Force  Commander  (CJTF).  The 
initial  focus  of  these  efforts  was  on  the  Battle  Watch  Captain  (BWC).  Members  of  the  DCD 
project  team  conducted  interviews  and  observed  exercises  and  actual  underway  operations 
onboard  USS  Coronado  (AGF  11)  and  USS  Mount  Whitney  (FCC  20)  as  part  of  the  CTA  effort. 

As  data  from  these  exercises  and  interviews  were  compiled,  experienced  cognitive  task  analysts 
examined  the  data  to  determine  decision  requirements,  information  flow  patterns,  training  and 
organizational  requirements,  and  common  operational  problems. 

The  initial  observations,  analyses,  and  interviews  quickly  revealed  that  the  CJTF,  Battle  Watch 
Captain,  and  the  supporting  personnel  are  not  well  served  by  current  JOC  information  systems 
and  workspaces.  Independently,  C2F  and  C3F  are  continuing  to  experiment  with  JOC  layouts, 
displays,  organizations,  and  decision  support  systems  as  they  evolve  toward  an  adequate 
configuration.  Clearly,  help  for  CJTF  is  urgently  needed.  Expected  products  from  the  DCD 
program  include  improved  information  management  and  display  systems,  and  recommendations 
for  changes  to  JOC  policies  and  procedures. 

1.  Introduction 

This  paper  discusses  lessons  learned  from  preliminary  CTA  conducted  on  JOC  personnel  aboard 
US  Navy  command  and  control  ships.  Based  on  lessons  learned,  recommendations  for  change 
are  provided.  The  CTA  was  performed  as  part  of  the  Navy’s  DCD  project.  The  information  and 
recommendations  contained  herein  are  preliminary  in  nature  and  subject  to  change  as  DCD 
project  efforts  continue. 

2.  Background 

Thanks  to  ongoing  advances  in  Information  Systems  technology,  today’s  military  tactical 
decision-makers  have  available  to  them  enormous  quantities  of  data.  Unfortunately,  having 


1  The  DCD  program  was  sponsored  by  OPNAV  N6,  and  managed  by  SPAWAR  PMW-133  and  SSC-SD  Code  44  during  the 
1998  fiscal  year.  Currently  in  hiatus  due  to  funding  constraints,  the  DCD  program  is  expected  to  resume  activity  in  FY  00. 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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access  to  more  data  does  not  always  help  decision-makers  perform  their  duties.  Instead,  what  is 
needed  is  readily  available,  accurate,  easy-to-understand  information. 

Current-generation  information  systems  and  displays  are  often  not  adequate  for  conveying 
tactical  information  to  Navy  decision-makers  -  especially  during  high  workload,  uncertain, 
ambiguous,  and  time  compressed  situations.  In  situations  like  these,  stress  can  have  very  serious 
effects  on  human  performance  and  decision  making  [Hutchins  and  Kowalski,  1993].  Inadequate 
information  and  display  systems  can  hinder  the  decision  making  process,  increase  the  potential 
for  blue-on-blue  and  blue-on- white  engagements,  and  negatively  impact  a  commander’s  ability 
to  operate  within  the  Rules  Of  Engagement  (ROE).  A  number  of  factors  further  exacerbate  the 
problem  [Kelly  and  Moore,  1996].  These  factors  include: 

•  Complex  and  sometimes  conflicting  mission  requirements,  often  in  Joint  or  Coalition 
settings. 

•  Situations  where  there  are  many  tracks  of  diverse  types  whose  status  as  friendly,  hostile 
or  neutral  is  ambiguous,  e.g.,  littoral  regions. 

•  Complex  and  ever-changing  orders  of  battle  and  ROE  associated  with  today’s  peace 
keeping  missions. 

•  Reduced  manning  requirements. 

Clearly,  improvements  are  needed  in  display  systems,  information  management  tools,  and 
human-computer  interfaces  (HCI).  Improved  systems  must  support  error-free  decision 
processes,  improve  situation  awareness,  improve  combat  team  interactions,  and  buy  time  for  the 
tactical  team  to  recognize  critical  events  and  operate  within  the  rules  of  engagement.  The  DCD 
project  is  working  toward  providing  such  improvements. 

3.  A  Proven  Approach:  Decision  Centered  Design 

One  method  for  improving  systems  and  displays  that  has  proven  quite  successful  in  the  past  is 
Decision  Centered  Design  (DCD).  Under  SSC-SD’s  current  DCD  efforts,  researchers  are 
working  to  develop,  document,  and  refine  a  rigorous,  formal  process  for  developing  information 
systems  tailored  to  the  requirements  of  decision  makers.  This  process  will  specify  necessary 
changes  in  system  development  and  acquisition  processes,  personnel  training,  and  doctrine  and 
procedures.  One  of  the  first  steps  in  the  DCD  process  is  to  develop  a  comprehensive 
understanding  of  all  aspects  of  a  particular  task  or  job.  To  do  this,  researchers  use  a  variety  of 
diverse  methods. 

3.1  Cognitive  Task  Analysis  (CTA) 

Conducting  a  Cognitive  Task  Analysis  (CTA)  [Klein  Associates  Inc.,  1997]  is  one  method  used 
to  develop  a  greater  understanding  of  a  particular  job  or  task.  This  type  of  analysis  attempts  to 
identify,  and  clearly  describe  the  cognitive  tasks  performed  by  a  decision-maker  performing  a 
specific  task.  One  especially  useful  tool  employed  in  CTA  is  an  interview-based  “knowledge 
audit.”  Knowledge  audits  -  a  specialized  form  of  structured  interview  -  can  be  performed  in 
formal  interviews  settings,  or  adapted  as  an  adjunct  to  other  knowledge  elicitation  /  data 
collection  methods. 


3.2  Structured  Observations 


Another  method  used  to  collect  information  about  a  particular  job  or  task  is  carefully  structured 
observations  [Moore,  1997].  Sometimes  time  sampling  techniques  are  applied  to  quantify  the 
data  collected  through  observations.  Other  times,  observations  are  made  on  an  occasional  or  one¬ 
time  basis,  and  then  validated  through  additional  observations  and  discussions  with 
knowledgeable  persons  such  as  the  actual  job  performer  or  subject  matter  experts  (SME). 
Researchers  carefully  observe  personnel  performing  specific  tasks.  During  the  observations, 
actions  are  recorded,  workspace  layouts  are  sketched,  and  supplementary  task  information  is 
collected. 

3.3  Findings  and  Recommendations 

This  report  provides  initial  findings  -  and  recommendations  based  on  these  findings  -  from  a 
partially  completed  CTA  conducted  on  the  Commander,  Joint  Task  Force  (CJTF)  Battle  Watch 
Captain  (BWC).  These  findings  and  recommendations  will  be  used  in  later  phases  of  the  DCD 
process  to  guide  researchers  and  engineers  in  the  development,  testing,  and  evaluation  of 
improved  information  management  and  display  systems. 

4.  Preliminary  Lessons  Learned  from  the  Joint  Operations  Center  (JOC)  Workspace  and 
Personnel 

The  following  information  is  compiled  from  discussions,  observations,  and  interviews  performed 
aboard  USS  Coronado  and  USS  Mount  Whitney  during  the  period  of  May  1998  to  January  1999. 
Additional  DCD  CTA  data  collection  is  planned,  and  results  from  these  efforts  will  be  published 
separately. 

4.1  Cognitive  Task  Analysis  (CTA)  Interviews 

The  CTA  data  provided  below  was  collected  in  both  formal  and  informal  settings.  Formal 
interviews  were  usually  conducted  while  in  port  or  during  periods  of  low-tempo  operations  while 
underway.  Unfortunately,  formal  interviews  were  often  difficult  to  obtain  because  BWC  /  JOC 
personnel  were  unavailable  for  formal  interviews  during  underway  periods  due  to  exercise 
tempo,  watch  schedules,  and  meeting  requirements.  Fortunately,  during  these  periods  the  BWC  / 
JOC  personnel  were  often  willing  to  be  interviewed  informally  (i.e.,  briefly,  for  a  couple  of 
minutes  at  a  time  -  usually  during  meals  or  after  evening  meetings).  Therefore,  the  majority  of 
CTA  data  represented  here  was  collected  informally.  Interviewees  included  senior  CJTF  staff, 
actual  BWC  watch  standers,  and  senior  and  junior  level  BWC  support  staff.  Questions  asked  by 
the  researcher  during  these  interviews  included: 

•  What  is  the  job  /  role  /  function  being  performed? 

•  What  “products”  are  produced  in  association  with  the  job? 

•  What  “job  smarts”  have  you  developed  to  aid  you  in  your  job? 

•  What  essential  “tipper”  information  do  you  notice  while  performing  your  duties? 


•  What  efforts  can  you  “off-load”  to  other  personnel?  What  efforts  are  just  too  important 
to  let  someone  else  do? 

•  What  is  difficult  about  the  job  /  role  /  function? 

•  With  regard  to  the  job,  where  does  the  most  valuable  information  come  from? 

•  If  you  could  change  your  display  /  system  in  any  way  you  want,  what  would  you  change? 

•  When  on  the  job,  who  do  you  communicate  most  with?  What  information  do  you 
exchange  with  this  person? 

•  What  distracts  you  when  the  workload  is  high? 

•  What  is  the  most  difficult  aspect  of  your  job,  especially  with  respect  to  decision-making? 

The  following  section  provides  representative  responses  these  interview  questions.  These 
responses  are  provided  for  informational  purposes  only.  Due  to  the  preliminary  nature  of  the 
findings  detailed  in  this  report,  and  the  fact  that  additional  information  collection  and  analysis 
are  ongoing,  the  following  information  should  be  considered  -  and  acted  upon  -  with  prudence. 

4.1.1  What  is  the  BWC’s  job  / role  /function ? 

There  was  a  great  deal  of  consensus  among  interviewees  with  regard  to  the  role  of  the  BWC. 
Interviewees  felt  that  the  primary  roles  of  a  BWC  were  to  act  as  Joint  Task  Force  “facilitator” 
and  represent  the  CJTF  in  day-to-day  operations.  They  agreed  that  the  BWC  was  responsible  for 
making  sure  that  all  of  the  diverse  Task  Force  resources  and  activities  were  coordinated, 
monitored,  and  reported  to  higher  authority.  Individual  responses  to  this  question  included: 

•  “Represent  the  Admiral”  during  day-to-day  operations . . . 

•  Advise  the  Admiral  on  operational  issues  (as  operational  plans  and  courses  of  action  were 
developed  and  put  into  place,  advise  the  admiral  with  regard  to  their  impact  on  mission 
goals  and  timelines). 

•  “Work  the  seams”  between  Component  Commanders  (make  sure  that  the  Component 
Commanders  were  coordinating  /  communicating  with  each  other,  and  that  their  activities 
did  not  conflict  with  each  other’s  activities  and  other  ongoing  events). 

•  Assist  Component  Commanders  with  resource  conflicts,  etc.  (work  to  resolve  conflicts 
with  regard  to  Task  Force  resource  management). 

•  Monitor  the  tempo  of  the  campaign  (monitor  all  ongoing  and  future  activities  as  they 
relate  to  the  overall  plan). 

•  Gather,  analyze,  fuse,  and  disseminate  operational  /  tactical  data  (build,  maintain,  and 
disseminate  a  common  “situation  awareness”  so  that  everyone  in  the  Task  Force  shares  a 
common  understanding  of  operational  /  tactical  issues). 

4.1.2  What  does  the  CJTF  need  /  expect  from  the  BWC  and  his  or  her  staff? 

Responses  regarding  the  actions  and  products  expected  from  the  BWCs  and  their  staff  indicate  a 
wide  range  of  tasking  -  from  mundane  administrative  duties  to  mission-critical  mission  planning 
and  monitoring.  “Products”  expected  from  the  BWCs  and  their  staff  include: 

•  Immediate  alerts  and  advise  regarding  any  of  the  CJTF’s  numerous  CCIRs  or  non- 
compliance  with  operational  guidance  (Commander’s  Intentions,  etc.) 


•  An  up-to-date  situation  awareness. 

•  A  clear  understanding  of  past,  current,  and  future  ops. 

•  A  “concept  of  ops”  /  recommendations  regarding  how  to  proceed  on  various  matters. 

4.1.3  What  “ job  smarts ”  have  you  developed  to  aid  you  in  your  duties  as  BWC? 

Due  to  widely  diverse  mission  tasking  and  a  constantly  evolving  concept  of  operations, 
respondents  stated  that  long-term  “job  smarts”  were  difficult  to  develop.  In  spite  of  this,  three 
general  “job  smart”  themes  emerged  from  these  discussions. 

•  Conduct  formal  watch  turnover  briefs  starting  30  minutes  prior  to  the  watch  to  ensure  that 
off-going  and  on-coming  watch  personnel  (and  any  other  interested  personnel)  have  a 
common  understanding  of  the  situation  and  the  Commander’s  intent. 

•  Maintain  close  working  relationships  with  other  JOC  personnel  and  embarked 
Component  Commander  staff. 

•  Make  effective  use  of  MS  Office.  (Few  JOC  staff  seemed  proficient  with  the  software 
suite) 

4.1.4  What  key  “ tipper ”  information  do  you  notice  while  performing  your  duties? 

Key  “tipper”  information  used  by  BWCs  (i.e.,  information  that  suggests  to  the  BWC  an 
upcoming  change  or  significant  event)  tended  to  be  somewhat  general  in  nature.  Changes  in 
established  track  behavioral  trends,  and  significant  changes  in  communications  between  own- 
force  and/or  enemy  units  are  examples  of  information  that  alert  the  BWC  to  impending 
significant  events.  As  reported  by  respondents,  key  “tipper”  information  indued: 

•  “Everything...”  (i.e,  the  BWC  attempted  to  consider  all  incoming  information  in  terms  of 
what  future  activity  it  might  suggest). 

•  Reports  from  other  ships 

•  Input  from  Intelligence  personnel 

•  Reports  from  other  ship’s  assets  (e.g.,  lookouts,  bridge  personnel,  sensor  operators,  etc.) 

•  Voice  traffic  on  the  command  circuit 

•  J7  (white  cell)  personnel  comments  (for  exercise  operations) 

•  Significant  air  activity  (enemy,  neutral,  and  friendly  activity  were  all  monitored) 

4.1.5  What  efforts  can  you  “off-load”  to  other  watch  personnel?  and  What  efforts  are  just 
too  important  to  let  someone  else  do? 

Most  BWC  watch-standers  stated  that  the  current  watch  structure  and  Standard  Operating 
Procedures  (SOP)  had  allocated  job  tasking  appropriately.  In  other  words,  the  tasks  that  they 
were  assigned  should  be  performed  by  the  BWC.  They  thought  that  it  was  inappropriate  to  off¬ 
load  tasks  to  junior  personnel.  The  only  exception  to  this  was  that  the  BWC  and  Assistant  BWC 
often  task-shared  as  necessitated  by  operational  tasking  and  tempo. 


4.1.6  What  is  difficult  about  being  a  BWC? 


Interviewees  indicated  that  there  were  a  number  of  difficulties  associated  with  performing  the 
job  of  BWC.  Most  of  the  comments  related  to  the  timeliness,  completeness,  quantity,  and 
quality  of  exchanged  information.  Responses  included: 

•  Working  with  time-late  information.  (Some  information  in  their  data  systems  was  three 
or  more  hours  old!) 

•  Being  informed  of  important  information  by  tactical  units  “after-the-fact.”  (They  weren’t 
being  kept  in  the  loop. . .) 

•  Too  many  communications  “circuits”  to  handle  (secure  and  unsecure  radio  channels, 
secure  and  unsecure  telephones,  e-mail,  required  face-to-face  communications,  etc.) 

•  Developing  and  maintaining  situation  assessment. 

•  Information  “disconnects”  with  Component  Commander  cells  aboard  ship  and  on  other 
units. 

•  Training  new  watch  personnel  and  “ramping-up”  staff  augmentees  from  other  units. 

4. 1 .7  Where  does  your  most  valuable  information  come  from? 

While  the  responses  to  this  question  varied  extensively,  the  underlying  theme  was  that  the  BWC 
considered  verbal  communications  with  support  staff  and  other  units  in  the  area  to  be  extremely 
valuable.  Specific  responses  to  this  question  included: 

•  JAC  (Joint  Analysis  Center,  i.e.,  the  Intell  LNO) 

•  Air  and  Ground  LNOs  (Liaison  Officers) 

•  Voice  traffic  across  the  command  nets  and  secure  phones 

•  JAG  (Judge  Advocate  General,  i.e.,  Legal  LNO) 

4.1.8  If  you  could  change  your  display  to  display  anything  you  want,  what  would  it  be? 

Interviewees  were  given  the  opportunity  to  provide  recommendations  for  improved  displays. 
This  question  often  generated  extensive  discussions  on  the  advantages  and  disadvantages  of 
various  systems,  however,  the  consistent  desire  was  for  more  (and  more  timely)  processed 
information ,  and  less  raw  data.  Representative  responses  included: 

•  Current  information...  (there  is  an  ongoing  issue  with  the  timeliness  of  data  presented  by 
some  data  systems  used  by  the  BWC) 

•  More  information...  (Many  respondents  felt  the  need  for  “more  information,”  however, 
they  weren’t  sure  what  exactly  they  would  want...) 

•  Composite  “cartoon”  /  Operational  Graphic  representing  S.A. 

4.1.9  Who  do  you  communicate  most  with  ( on-ship  or  off-ship)?  and  What  information  do 
you  exchange  with  this  person? 

As  evidenced  by  the  responses  to  the  question  “Where  does  your  most  valuable  information 
come  from?”  BWC  personnel  indicated  that  communications  with  their  support  staff  and  other 


units  in  the  area  were  extremely  important.  As  a  result,  they  communicated  with  these  entities  on 
a  regular  basis.  Listed  below  are  the  most  common  responses  regarding  the  entities 
communicated  with,  and  the  types  of  information  exchanged  with  them. 

•  LNOs  (Liaison  Officers)  within  the  JOC 

-  New  information,  recommendations,  updates,  etc. 

-  Representatives  from  other  component  commanders 

-  Updates  regarding  current  and  future  operations 

•  Other  units  in  the  area. 

-  Information  regarding  ongoing  and  past  events 

-  Requests  for  direction  /  authorization  regarding  various  actions 

4.1.10  What  distracts  you  when  the  workload  is  high? 

Responses  to  queries  regarding  distractions  to  the  BWC  varied  along  a  number  of  dimensions. 
Some  distractions  were  operationally  relevant  (e.g.,  the  need  to  monitor  numerous 
communications  nets  simultaneously),  while  others  were  more  mundane  (e.g.,  the  heating  /  air 
conditioning  system  not  adequately  controlling  the  ambient  air  temperature  in  the  JOC). 
Examples  of  observed  and  stated  distractions  within  the  JOC  are  included  below. 

•  The  need  to  get  the  next  briefing  ready. . .  (There  were  five  major  briefings  per  day!) 

•  Ambient  noise  within  the  JOC 

•  C2F  JOC  is  co-located  with  a  VTC  center  and  a  primary  meeting  area,  thus  ambient  noise 
level  is  regularly  quite  excessive. 

•  Ambient  temperature  within  the  JOC 

•  A  less-than-optimized  A/C  system  resulted  in  a  work  space  that  was  often  too  hot,  or  too 
cold. 

•  The  need  to  monitor  numerous  communications  circuits  simultaneously 

•  Constant  interruptions  by  C2F  non-watch  standing  staff 

4.1.11  What  is  the  most  difficult  aspect  of  your  job,  especially  with  respect  to  decision¬ 
making? 

There  seemed  to  be  a  great  deal  of  consensus  regarding  to  the  most  difficult  aspect  of  acting  as 
BWC.  Specifically,  gathering,  analyzing,  fusing,  and  disseminating  all  operational  information 
available;  and  using  this  information  to  develop  and  maintain  an  accurate,  up-to-date  Situation 
Awareness  was  considered  by  far  the  most  difficult  aspect  of  the  job. 

4.2  Observations 

Observations  of  BWC  personnel  and  their  workspaces  were  made  during  several  visits  to  the 
USS  Coronado  (AGF  11)  (see  Figure  L),  and  USS  Mt  Whitney  (FCC  20)  (see  figure  2.).  These 
visits  occurred  while  the  ships  were  underway  and  in  port.  As  observations  were  made, 
comments  were  captured  and  drawings  /  sketches  created.  This  information  has  been  reduced  and 
combined,  and  is  presented  below  in  “bulletized”  summary  form. 


4.2.1  Observations  from  USS  Coronado  (AGF 11) 

•  Anchor  Desks  need  to  provide  “value  added”  information  to  the  BWC  (i.e.,  graphical 
representations  of  relevant  data) 

•  Anchor  Desks  need  ability  to  effectively  monitor  tactical  /  operational  displays 

•  BWC  needs  assistance  defining  and  displaying  information  dealing  with  operational 
issues 

•  Augmentee  roles  &  responsibilities  need  to  be  better  defined  and  promulgated 

•  Augmentee  staff  need  to  understand  their  roles 

•  Focus  needs  to  be  on  relevant  tasks 

•  Watch  standers  need  to  have  a  common  understanding  of  assigned  tasks 

•  Large  displays  used  for  sharing  information  unreadable  /  difficult  to  use 

1.  Difficult  to  read  from  a  distance  (inadequate  resolution  &  font  size) 

2.  Screen  format  /  layout  not  optimized  for  intended  task 

•  Number  of  communications  circuits  (and  the  need  to  monitor  and  interact  on  them) 
sometimes  interfered  with  other  duties. 

•  Number  of  non-watch  standing  personnel  in  (or  passing  through)  the  JOC  sometimes 
negatively  effected  BWC  duties 


Large 

Screen 

Displays 


Figure  1.  USS  Coronado  JOC  -  after  recent  re-design 


4.2.2  Observations  from  USS  Mount  Whitney  (LCC  20) 


•  Most  JOC  personnel  spend  the  vast  majority  of  their  time  (perhaps  as  much  as  50%) 
using  MS  Office  to  perform  various  functions  (though  few  seem  to  have  adequate 
training  with  the  software  suite...) 

•  Tactical  /  operational  display  use  (ADSI,  JMCIS,  and  MCCIS)  was  sometimes  not 
optimal 

•  Need  to  “sanitize”  information  for  use  by  multi-national  staff  was  a  significant  burden; 
manual  “sanitation”  often  delayed  data  by  as  much  as  three  or  more  hours 

•  Air  Defense  Systems  Integrator  (ADSI)  system  in  the  JOC  sometimes  displayed 
incomplete  data,  however,  ADSI  in  JFACC  Current  Ops  was  extremely  accurate  and 
timely.  The  disparity  between  the  JOC  ADSI  picture  and  other  ADSI  pictures  was 
apparently  the  result  of  inadequate  training  of  the  ADSI  operators  within  the  JOC 
(LINK/comms  problems  also  contributed) 

•  SATCOM  circuits  (the  primary  means  of  secure  communications  for  the  JOC)  were  often 
unavailable  or  unreliable 

•  Important  information  was  routinely  transferred  by  floppy  disk  due  to  the  need  to  keep 
the  US  SECRET  (SIPRNET),  NATO  SECRET,  NATOLAN,  and  US  Unclassified 
(NIPRNET)  LANs  separate 

•  There  seemed  to  be  occasional  “information  disconnects”  between  the  JOC  and 
component  commanders  aboard  ship  and  on  other  units 

•  A  new  (and  very  expensive)  air  conditioning  system  dramatically  reduced  one  source  of 
ambient  noise  in  the  JOC. 

•  MCCIS,  ADSI,  and  JMCIS  offer  user  definable  color  settings.  As  a  result,  each  system 
had  different  color  coding  conventions,  therefore  users  could  not  easily  use  each  other’s 
displays... 

•  The  large  screen  displays  (LSDs)  (in  USS  Mt.  Whitney’s  case,  37”  television  monitors) 
could  be  programmed  to  show  any  display  within  (or  outside)  the  JOC.  There  was  no 
standard  set-up  for  what  information  was  to  be  presented  where.  As  a  result,  the  LSDs 
presented  different  information  literally  every  few  minutes... 

•  VTC  communications  ranged  from  “rock  solid”  to  completely  unusable... 
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Figure  2.  USS  Mt.  Whitney  JOC  -  after  recent  re-design 

5.  Preliminary  Recommendations  Based  on  Lessons  Learned 

Preliminary  analyses  indicated  problems  in  preparing  and  disseminating  coordinated  action 
plans,  in  developing  shared  awareness  of  the  tactical  situation,  in  maintaining  communications 
connectivity,  in  managing  tactical  responses  and  battle  force  assets,  and  even  in  understanding 
the  roles  and  responsibilities  of  the  CJTF/JOC  staff.  Areas  for  improvement  in  CJTF/JOC 
operations  were  prepared  as  general  recommendations  for  system  improvements  (i.e.,  not  as 
finger-pointing  evaluations)  and  discussed  with  key  personnel  at  OPNAV  and  fleet  commands. 
The  areas  for  improvement  were  acknowledged,  and  our  recommendations  were  greeted 
favorably  by  the  admirals.  These,  then,  will  become  the  basis  for  the  development  of  enhanced 
decision  support  display  concepts  that  are  to  be  prototyped  and  tested  in  the  DCD  laboratory 
under  realistic,  demanding  CJTF  scenarios.  Storyboards  for  preliminary  designs  were 
developed,  leading  to  a  "data  wall"  concept  as  a  means  for  collaborative  problem  solving  in  the 
JOC.  In  addition,  suggestions  were  made  for  "quick  fixes"  that  could  be  easily  made  within  the 
command  ship  JOCs  that  would  address  some  aspects  of  fundamental  information  management 
problems  revealed  by  the  cognitive  task  analysis  process.  Problem  areas  and  associated 
recommendations  are  discussed  below  in  “bulletized”  form. 


5 . 1  “Service”  Related  Issues 


5.1.1  Product  Focus 

•  Need  to  make  watch  turnover  and  daily  briefings  more  focused  (operationally  relevant)  - 
emphasis  on  “future  focus” ! ! ! 

Recommendation: 

—  Develop  PowerPoint  and  Word  “templates”  to  aid  BWC  create  operationally  relevant 
presentations  and  pass  down  logs. 

•  Recent  Events  as  they  relate  to  current  and  future  ops 

•  Conflicts  / problems  that  must  be  resolved 

•  Planned  /  upcoming  events 

5.1.2  “Customer  Service” 

•  Different  “customers”  need  different  levels  of  detail  /  fidelity  /  depth  of  information 
Recommendation: 

—  Conduct  analysis  of  customer  base  and  identify  similar  and  different  information 
requirements  (e.g.,  the  Admiral,  Component  Commanders,  and  operational  units) 

Some  customers  need  concise  “overview  data”  and  recommendations  for  action 
Some  customers  need  extremely  detailed,  “give-and-take”  discussion  of  issues 
Some  customers  need  orders  /  directions 

Information  Display  /  Visualization 

5.2.1 


•  BWC’s  need  a  way  to  visualize  the  “whole  mission.”  Goals,  resources,  plans,  actions, 
timelines,  etc.  must  be  represented  so  that  a  BWC  /  CJTF  can  tell  where  they  are  in  the 
“Grand  Plan”  and  answer  questions  like  “How  are  we  doing?”  and  “What  is  our  status?” 

Recommendation: 

—  Develop  a  composite  display  (data  wall?)  that  brings  together  relevant  mission  details 
—  Develop  “intelligent  agents”  to  identify  conflicts  /  problems  and  alert  user 
—  Develop  “Summary  Displays”  for  BWC 

5.2.2  Tower  of  Babel... 

•  Sharing  a  “common  picture”  across  systems  and  units  is  difficult  to  achieve,  and  it  is 
difficult  to  determine  whether  it  is  truly  “common” 

Recommendation: 

—  Ensure  all  units  have  similar-capability  data  systems  and  compatible  display  systems 
(easier  said  than  done!!) 


5.2.3  Data  Reduction,  Organization,  Presentation 


•  During  watch,  need  to  distill  huge  amounts  of  data  into  focused,  operationally  relevant 
S.A. 

Recommendation: 

—  Develop  “intelligent  agents”  that  aid  the  user  in  reducing  /  de-emphasizing  extraneous 
data  and  optimizing  the  presentation  of  needed  information 
—  Augment  existing  displays  with  graphically  represented  information  (e.g.,  overlays, 
future  states,  resource  conflicts,  etc.) 

5.2.4  “A  picture  is  worth  a  thousand  words ...” 

•  Need  to  build  and  maintain  easily  understood  “cartoon”  graphics  (using  PowerPoint)  to 
convey  S.A. 

Recommendation: 

—  Develop,  and/or  provide  tools  to  aid  BWC  in  the  rapid  creation  of  operationally 
relevant,  easy-to-understand,  composite  /  summary  graphics  that  incorporate  multiple 
information  domains  (e.g.,  include  integrated  METOC  and  air  operations  and  ground 
operations  and  Intelligence  information) 

•  PowerPoint  “templates” 

•  map,  chart,  and  icon  libraries 

•  easy-to-use  graphic  tools  ( perhaps  COTS?) 

5.2.5  Putting  the  Picture  Together 

•  Reliance  on  PowerPoint  slides  is  a  little  bit  of  a  “too  much  of  a  good  thing”  situation...  It 
is  an  extremely  valuable  asset  but  consumes  far  too  much  of  their  time!  (viewing,  creating, 
editing) 

Recommendation: 

—  Create  clear  guidelines  for  briefings  (e.g.,  no  more  than  X  number  of  slides,  font  size 
should  be  Y,  use  template  Z,  etc.) 

—  Address  perception  that  every  brief  is  a  full-scale  “command  performance”  requiring 
hours  of  preparation  -  instead,  focus  on  What  has  changed?  and  What  are  the 
implications  to  future  events  /  plans? 

5.2.6  “What  color  is...?” 

•  MCCIS,  ADSI,  and  JMCIS  offer  user  definable  color  settings.  As  a  result,  each  system 
often  has  different  color  coding  conventions,  thus  users  can  often  not  use  or  understand 
each  other’s  displays... 

Recommendation: 

—  Adopt  standard  Human  Factors  guidelines  and  principles  with  regard  to  color  selection 
—  Enforce  common  use  of  colors  across  all  systems 
—  Develop  and  implement  training  with  regard  to  adopted  color  sets 


“ Where  do  I  look  for. 

•  The  large  screen  displays  (LSDs)  could  be  programmed  to  show  any  display  within  (or 
where.  As  a  result,  the  LSDs  often  presented  different  information  at  various  points 

Recommendation: 

—  Standardize  use  of  LSDs  (e.g.,  METOC  is  always  on  Display  X)  so  that  users  always 
know  where  to  look  for  desired  information 
Develop  and  promulgate  ship’s  doctrine  dictating  LSD  use 

LSD  Problems 

•  Large  displays  used  for  sharing  information  unreadable  from  a  distance  (inadequate 
resolution  &  font  size) 


displays  (Data  Wall?) 

-  Re-arrange  JOC  layout  so  that  users  are  closer  to  displays 
5.2.9  ?!” 

•  Anchor  Desks  need  ability  to  effectively  monitor  tactical  /  operational  displays  (i.e.,  keep 
them  in  the  loop...) 


their  use 

—  Encourage  use  of  composite  “cartoon”  graphics  that  integrate  information  from 
multiple  domains 

Use  single  display  as  focal  point  for  discussion  and  resolution  of  mission-relevant 


5.2.10  AD  SI  Display  Use 

Air  Defense  Systems  Integrator  (ADSI)  display  in  the  JOC  sometimes  displayed 

Whitney))  was  extremely 

Recommendation: 

—  Consider  ADSI  for  use  as  primary  tactical  display 
—  Ensure  users  are  fully  trained  on  its  capabilities  and  limitations 


5.3  Hardware  / Software 


5.3.1  Technology 

•  There  is  a  perception  by  some  that  there  are  too  many  “gadgets”  in  the  JOC ! ! ! !  Chronic 
problems  with  various  technologies  require  a  dedicated  “technology”  watch  stander  in  the 
JOC  at  all  times  (which  they  currently  lack...)  COUNTERPOINT:  There  are  too  many 
people  in  the  JOC  already. . . 

Recommendation: 

—  Use  reliable  “tested  and  approved”  HW  /  SW 

—  Ensure  users  are  fully  trained 

—  Designate  a  “technology”  watch-stander  -  outside  the  JOC  -  to  handle  problems  as  they 
arise 

5.3.2  Microsoft  Office 

•  Most  JOC  personnel  spend  the  vast  majority  of  their  time  (sometimes  upwards  of  50  to 
60%)  using  MS  Office  to  perform  various  functions.  Surprisingly,  few  JOC  personnel 
seem  to  have  adequate  training  with  the  software  suite... 

Recommendation: 

—  Encourage  the  use  of  Command-approved  templates  to  facilitate  application  use 

—  Ensure  personnel  are  fully  trained  on  MS  Office  capabilities  and  limitations 

5.4  “Where  is  the  Value  Added?” 

•  Anchor  Desk  /  Liaison  personnel  must  provide  “value  added”  information  to  the  BWC 
(i.e.,  useful  /  graphical  representations  of  relevant  data) 

Recommendation: 

—  Conduct  analysis  to  determine  BWC  information  requirements  (vs.  data 
requirements...) 

-  Conduct  training  to  ensure  support  personnel  understand  BWC  needs,  goals,  tasking, 
etc. 

—  Develop  /  Provide  tools  to  help  the  Anchor  Desk  /  Liaison  personnel  portray  the 
information  they  share  in  the  context  of  the  whole  mission,  i.e.,  help  “connect  the  dots” 

5.5  Communications  &  Information  Exchange 

5.5.1  Reliance  on  the  Unreliable 

•  SATCOM  circuits  (the  primary  means  of  secure  communications  for  the  JOC)  were  often 
unavailable  or  unreliable 


Recommendation: 

—  Explore  other  secure  means  of  communicating 

•  e-mail 

•  COMPASS 

•  NetMeeting 

—  Develop  guidelines  for  controlled  use  of  SATCOM  (instead  of  using  it  to  discuss  non¬ 
sensitive  topics  /  issues) 

5.5.2  “Who’ s  on  First?” 

•  There  seemed  to  be  regular  “information  disconnects”  between  the  JOC  and  Ship  / 
Component  Commanders  aboard  ship  and  on  other  units 

Recommendation: 

—  Conduct  analysis  to  determine  causes  of  information  disconnects 
—  Establish  and  promulgate  guidelines  to  ensure  that  vital  information  is  shared  in  a 
timely  manner 

5.5.3  Communication  -  at  what  cost? 

•  Number  of  communications  circuits  (and  the  need  to  monitor  and  interact  on  them) 
sometimes  competed  with  /  conflicted  with  other  tasks  and  duties. 

Recommendation: 

—  Consolidate  communications  circuits  within  JOC  (i.e.,  instead  of  multiple  secure 
channels,  use  one) 

—  Delegate  communications  duties  to  subordinate  personnel 

—  Re-direct  non-essential  communications  to  other  “channels”  (e.g.,  e-mail,  standard 
message  traffic,  etc.) 

5.5.4  VTC:  Cutting  Edge  Technology 

•  VTC  communications  ranged  from  “rock  solid”  to  completely  unusable... 
Recommendation: 

—  Use  VTC  for  non-essential  communications  only  until  reliability  is  improved 
—  Transmit  PowerPoint  presentations  via  e-mail  before  briefing,  then  use  standard  voice 
comms  to  talk  thru  slides  (instead  of  using  VTC  to  simply  display  slides) 

5.5.5  Headset  Pros  and  Cons 

•  Headsets  might  help  with  certain  communication  problems  in  the  JOC 
COUNTERPOINT:  Headsets  would  negatively  impact  verbal  communications  between 
JOC  personnel 

Recommendation: 

—  Explore  use  of  COTS  “low-profile,”  “non-exclusive”  headset  technology  that  allows 
user  to  hear  ambient  sounds  normally,  yet  still  hear  communications  through  headset 


5.5.6  AT&T  Phone  Operator? 


•  Too  many  people  call  the  JOC  for  non-operation  related  reasons!!  (for  example,  during 
one  watch,  an  Ensign  was  called  from  the  beach  by  his  wife/  girlfriend  to  talk. . .  and  the 
Ensign  was  not  even  assigned  as  a  JOC  watch- stander) 

Recommendation: 

—  Do  not  allow  non-JOC-related  calls  to  be  taken  in  the  JOC 
—  Do  not  publish  JOC  phone  numbers  for  general  use 

5 .6  Miscellaneous  (. Policies  /  Procedures  /  Training  /  etc.) 

5.6.1  Roles  and  Responsibilities 

•  Staff  (especially  augmentees)  roles  &  responsibilities  need  to  be  better  defined  and 
promulgated 

•  The  focus  needs  to  be  on  relevant  tasks 

•  Watch  standers  need  to  have  a  common  understanding  of  assigned  tasks 
Recommendation: 

—  Develop  and  implement  training 

5.6.2  Access  Control 

•  Number  of  non-watch  standing  personnel  in  (or  just  passing  through)  the  JOC  sometimes 
had  a  negative  effect  on  BWC  duties 

Recommendation: 

—  Reduce  access  to  JOC 
—  Implement  procedures  to  control  access 

5.6.3  Blessing  and  Burden:  Multi-National  Staff  (Mt  Whitney ) 

•  Need  to  “sanitize”  information  for  use  by  multi-national  staff  was  a  significant  burden  - 
manual  “sanitation”  often  delayed  data  transfer  of  data 

•  Important  information  was  routinely  transferred  via  floppy  disk  due  to  the  need  to  keep 
classified  and  unclassified  LANs  separate  (MT  Whitney) 

Recommendation: 

—  Develop  and  implement  automated  information  sanitation  agents 

-  Develop  “traffic  cop”  software  to  allow  classified  and  unclassified  material  to  reside  on 
the  same  network 

5.6.4  Simple  Matter  of  Perception  Management  -  Or  a  Real  Problem? 

•  The  is  a  perception  by  some  that  BWCs  should  be  0-6  level  officers  (currently  they  are  O- 
4s  and  0-5s) 

Recommendation: 

—  Conduct  analysis  to  determine  actual  task  requirements 


5.6.5 


containing  non-operationally  relevant  information  regarding  the  day’s  events) 
Recommendation: 

Develop  /  Adapt  voice  activated  log  entry  technologies  (e.g., 


6.  Summary 


Military  decision  makers  are  now  routinely  able  to  communicate  instantly  in  several  media  with 
others  around  the  globe,  locate  and  combine  huge  amounts  of  data  from  diverse  sources,  and 

these  technical  advances  have  been  substantial,  the  corresponding  expected  improvement  in 
decision-making  performance  has  been  disappointing.  One  reason  for  this  is  that  new  C  ISR 
systems  tend  to  be  developed  based  on  the  latest  technology  with  little  regard  for  the  decision- 


In  contrast,  decision-centered  design  (DCD)  focuses  on  the  cognitive  functions  and  tasks  that 
experienced  decision-makers  perform  in  real  situations.  Information  support  and  communication 

capabilities  can  then  be  assembled  to  meet  the  decision  requirements  that  have  been  identified. 

The  DCD  process  focuses  advanced  technology  on  serving  the 

than  forcing  decision  makers  to  adapt  to  the  capabilities  and  procedures  of  new  systems. 

As  a  result  of  the  preliminary  CTA  efforts  described  in  this  paper,  SSC-SD  researchers  now  have 

display  systems  to  aid  the  decision-maker  in  difficult  decision-making  domains. 
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